Input Device for Commanding Control Operations for 
Motions of a Real or Virtual Object 

BACKGROUND DESCRIPTION 

Field of the Invention 

[0001] The invention relates to a manually actuated input 
device for commanding machine- and/or computer-assisted 
control operations for kinematic motions of a real or 
virtual multipart object, including a force/moment sensor 
with which linear displacements in the form of translational 
movements in the direction of three axes, each standing 
perpendicular on the other, of a three-dimensional 
rectangular system of coordinates and/or rotational 
excursions in the form of rotational motions about these 
three axes are sensed and converted into commanded motions 
of the object to be controlled. 

[0002] In addition, the invention relates to possible 
applications of the input device. 

Prior Art 

[0003] Such force/moment sensors are known, for example 
from US 4,763,531 and US 4,785,180. With the aid of one such 
force/moment sensor, accommodated in an input device, linear 
displacements and/or rotational excursions, i.e. three 
translational movements in the direction of three axes (X, 
Y, Z) , each standing perpendicular on the other, of a three- 
dimensional rectangular system of coordinates and three 
rotational motions about these three axes, can be sensed and 
directly converted into commanded translational and 
rotational motions or velocities of the complete object to 
be controlled, e.g. for the control of automated and 



robotic, manipulator or similar systems as well as 3D 
computer graphics . 

[0004] Modern computers are becoming increasingly more 
sophisticated in implementing complex real-time 3D motional 
control, requiring manipulation of the motions and possibly 
changing the sounds associated therewith to be controlled in 
accordance with human appreciation. 

[0005] Known in this context from US 5,757,360 A is a 
manually actuated input device for controlling a virtual 
object in a computer display. In this arrangement, the 
object itself can be moved in accordance with the movement 
of the egg-shaped manual input device, or dedicated motion 
patterns of the input device can be caused to action parts 
of the virtual object. The input device used for this 
purpose is based on the principle of an accelerometer which 
recognizes accelerations and patterns thereof from the 
motions implemented in predefined patterns of the complete 
manual input device in allocating a motion characteristic 
assigned to each acceleration or pattern to the object shown 
in the display. In this known input device provided only for 
controlling and animating virtual, but not real, objects, 
i.e. robots and figures, difficulties are encountered, 
however, in correctly implementing a manual movement 
assigned to a specific, desired acceleration pattern, 
especially when the object needs to be animated in a wealth 
of different variants. This is due to the motion of egg- 
shaped input device having no fixed reference base in free 
space for precise guidance and because only acceleration 
reactions are evaluated. Accordingly, this known input 
device fails to achieve animation of the virtual object with 
high diversification due to its low targetting accuracy, and 
thus the animation is lacking in realistic and natural 
presentation . 



Summary of the Invention 



[0006] The invention is based on the objective of 
sophisticating an input device of the aforementioned kind so 
that defined and intuitive implementation of kinematic 
control and animation operations of objects is possible. The 
objects to be controlled or animated are intended to be more 
particularly human-like robots or computerized virtual 
animate beings, whilst achieving high target accuracy of 
each of a wealth of desired control or animation variants. 

[0007] In accordance with the invention, relating to an 
input device of the aforementioned kind, this objective is 
achieved in that the commanded individual linear 
displacements and/or rotational excursions of the 
force/moment sensor are assigned specific kinematic motion 
patterns of parts of the real or virtual object thereby 
permitting manipulation or animation thereof as a 
transforming interlink and in that the commanded velocities 
of the corresponding individual linear displacements and/or 
rotational excursions of the force/moment sensor are 
additionally weightable as a kinematic interlink assignment. 
In this arrangement the individual motions of the kinematics 
to be controlled are transformed similar to the way in which 
a player manipulates a puppet . 

[0008] Advantageous applications of the input device in 
accordance with the invention involve machine- or computer- 
controlled objects simulated by inherently and/or totally 
movable objects or animate beings, whereby parts of the 
object or animate being, e.g. the arms, legs, trunk and head 
of an animate being, simulated in each case, form the parts 
of the object to be controlled as regards their kinematics. 
Its use for computer-controlled animation operations of the 
objects is also expedient. 



[0009] Since the input device in accordance with the 
invention permits simultaneous control of up to six degrees 
of freedom, motions and animations of machine- and/or 
computer-controlled simulated animate beings or objects are 
represented highly realistically and naturally. 

[0010] Since for six degrees of freedom there are twelve 
major directions of motion, to each of which an interlinking 
assignment can be allocated, these major directions of 
motion can be additionally sub-related by rates and/or 
positions so that for two rates and/or positions 24 
kinematic assignments would already be possible. For three 
rates and/or positions this would be 36 kinematic 
assignments, etc. 

[0011] The input device in accordance with the invention is 
not only solely applicable to controlling the motions of 
parts of the object. Instead, in addition to controlling the 
motions of parts of the object it is possible to advantage 
to combine and/or vary tones and/or music of any kind. This 
would mean, for example, that when the operator is 
controlling a higher velocity of the input device, tones 
such as e.g. walking noises or music sequences sound and/or 
change in thus enabling friction sounds or also dramatic 
music sequences to be combined with the motional control in 
effectively enhancing the reality and naturalness. 

[0012] The configuration for achieving one particularly 
advantageous input device including a force/moment sensor is 
set forth in claim 7 and in the claims 8 to 18 relating back 
thereto directly or indirectly. These or similar 
force /moment sensors are known, for example, from US 
4,763,531 or US 4,785,180. With the aid of one such 
force/moment sensor accommodated in an input device, linear 
displacements or rotational excursions, as may be generated 
e.g. by the human hand in the form of forces and moments, 
can be sensed and directly converted into translational and 
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rotational motions or velocities of an object to be 
controlled . 

[0013] Since the invention in making use of a force/moment 
sensor, e.g. of the aforementioned kind, relates to an 
analog senso-haptic input device, combining e.g. an 8 -bit 
sensor resolution of 512 {± 256) rates and 512 (i 256) 
changes in tone/sound with the motional control, it permits 
achieving a wealth of possible combinations. 

Brief Description of the Drawings 

[0014] The invention will now be detailled with reference 
to the attached drawings in which: 



Fig. 1 is an inclined view of six possible motional 

components of a force/moment sensor used in the 



input 



device in accordance with the invention, 



Figs. 2 to 20 are diagrammatic illustrations of various 

assignments between motions of a force/moment sensor 
and motions of a simulated human. 



Fig. 21 is a diagrammatic illustration of an input device 
containing an optoelectronic force/moment sensor 
including a manual control cap shown cut open in 

part , and 

Fig. 22 is a diagrammatic illustration of the optoelectronic 
assembly accommodated in the input device as shown 

in 

Fig. 21. 



Detailled Description 



[0015] Referring now to Fig. 1 there is illustrated how six 
possible motional components can be sensed by an input 
device. The input device, as shown by way of example in 
Figs. 21 and 22, including a force/moment sensor, is capable 
of sensing these motional directions in sensing all linear 
displacements and rotational excursions as may be generated 
by the human hand in the form of forces and moments for 
implementing control of an object and/or parts thereof. 

[0016] The possible linear displacements run in both 
directions of the three axes X, Y and Z of a three- 
dimensional, rectangular system of coordinates each 
indicated by double arrows, whilst the rotational 
excursions, i.e. the moment vectors A, B and C generated 
about each coordinate axis X, Y and Z are indicated by 
arrowed sectors. The forces and moments measured by the 
input device, e.g. that as shown in Figs. 21 and 22 are 
directly converted into translational and rotational motions 
and velocities of the individual machine- or computer- 
controlled parts of the object. 

[0017] Referring now to Figs. 2 to 20 there are illustrated 
diagrammatical ly by way of pictograms how various 
conceivable assignments between motions of a force/moment 
sensor and motions of a simulated human are achievable 
involving the directions as shown in Fig. 1. The 
force/moment sensor used for this purpose may be, more 
particularly, one such force/moment sensor as shown in Figs. 
21 and 22 which will now be described with reference 
thereto . 

[0018] Slowly pulling up the grip control cap (8 in Fig. 
21) from its starting position, i.e. a slow movement in the 
positive Z direction, results in up and down movement of the 
arms (= parts of the object) , as indicated by the pictogram 
representing a human (= object) in Fig. 2. 
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[0019] Quickly pulling up the grip control cap (8 in Fig. 
21) from its starting position, i.e. a quick movement in the 
positive Z direction, results in the object jumping with 
suitable movement of the arms and legs, as indicated by the 
pictogram representing a human in Figs. 3 and 4. 

[0020] Slowly depressing the grip control cap (8 in Fig. 
21) from its starting position, i.e. a slow movement in the 
negative Z direction, results in the object assuming a 
seating position with suitable movement of the parts of the 
body, as indicated by the pictogram representing a human in 
Fig. 5. 

[0021] Quickly depressing the grip control cap (8 in Fig. 
21) from its starting position, i.e. a quick movement in the 
negative Z direction, results in the object doing the splits 
with suitable movement of the parts of the body, as 
indicated by the pictogram representing a human in Fig. 6. 

[0022] Slowly shifting the grip control cap (8 in Fig. 21) 
forwards from its starting position, i.e. a slow movement in 
the positive Y direction, results in the object walking 
forwards with suitable movement of the parts of the body, as 
indicated by the pictogram representing a human in Fig. 7. 

[0023] Quickly shifting the grip control cap (8 in Fig. 21) 
forwards from its starting position, i.e. a quick movement 
in the positive Y direction, results in the object running 
forwards with suitable movement of the parts of the body, as 
indicated by the pictogram representing a human in Fig. 8. 

[0024] Slowly shifting the grip control cap (8 in Fig. 21) 
backwards from its starting position, i.e. a slow movement 
in the negative Y direction, results in the object walking 
backwards with suitable movement of the parts of the body, 
as indicated by the pictogram representing a human in Fig. 
9 . 
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[0025] Quickly shifting the grip control cap (8 in Fig. 21) 
backwards from its starting position, i.e. a quick movement 
in the negative Y direction, results in the object running 
backwards with suitable movement of the parts of the body. 
The pictogram in this case (not shown) is similar to that as 
shown in Fig. 9, although, of course, the movement sequence 
would be quicker. 

[0026] Slowly shifting the grip control cap (8 in Fig. 21) 
to the right from its starting position, i.e. a slow 
movement in the positive X direction, results in the object 
side-stepping to the right with suitable movement of the 

If^ parts of the body, as indicated by the pictogram 

K representing a human in Fig. 10. 

m 

fjj [0027] Quickly shifting the grip control cap (8 in Fig. 21) 

to the right from its starting position, i.e. a quick 
e movement in the positive X direction, results in the object 

g jumping to the right with suitable movement of the parts of 

rV the body, as indicated by the pictogram representing a human 

in Fig. 11. 

P 

[0028] Slowly shifting the grip control cap (8 in Fig. 21) 
to the left from its starting position, i.e. a slow movement 
in the negative X direction, results in the object side- 
stepping to the left with suitable movement of the parts of 
the body, as indicated by the pictogram representing a human 
in Fig . 12 . 

[0029] Quickly shifting the grip control cap (8 in Fig. 21) 
to the left from its starting position, i.e. a quick 
movement in the negative X direction, results in the object 
jumping to the left with suitable movement of the parts of 
the body, as indicated by the pictogram representing a human 
in Fig. 13 . 



[0030] Slowly turning the grip control cap transversely to 
its running direction, i.e. a slow positive turn about the X 
direction, results in the head of the object nodding down, 
as indicated by the pictogram representing a human in Fig. 
14 . 

[0031] Quickly turning the grip control cap transversely to 
its running direction, i.e. a quick positive turn about the 
X direction, results in the object making a bow, as 
indicated by the pictogram representing a human in Fig. 15. 

[0032] Slowly turning the grip control cap transversely to 
its running direction, i.e. a slow negative turn about the X 
direction, results in the head of the object nodding 
upwards, as indicated by the pictogram representing a human 
in Fig . 16 . 

[0033] Slowly turning the grip control cap about its 
running direction, i.e. a slow positive turn about the Y 
direction, results in the head of the object nodding to the 
right, as indicated by the pictogram representing a human in 
Fig. 17. 

[0034] Quickly turning the grip control cap about its 
running direction, i.e. a quick positive turn about the Y 
direction, results in the object swaying to the right, as 
indicated by the pictogram representing a human in Fig. 18. 

[003 5] Slowly turning the grip control cap about its 
running direction, i.e. a slow negative turn about the Y 
direction, results in the head of the object nodding to the 
left, as indicated by the pictogram representing a human in 
Fig. 19. 

[0036] Quickly turning the grip control cap about its 
running direction, i.e. a quick negative turn about the Y 
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direction, results in the object swaying to the left, as 
indicated by the pictogram representing a human in Fig. 20. 

[0037] Slowly turning the grip control cap about the body 
axis of the object, i.e. a slow positive turn about the Z 
direction, results in the object twisting to the right. 
Quickly turning the grip control cap about the body axis of 
the object, i.e. a quick positive turn about the Z 
direction, results in the object making a twisting jump to 
the right . Slowly turning the grip control cap about the 
body axis of the object, i.e. a slow negative turn about the 
Z direction, results in the object twisting to the left. 

[003 8] Quickly turning the grip control cap about the body 
axis of the object, i.e. a quick positive turn about the Z 
direction, results in the object making a twisting jump to 
the left.- No pictograms of a human are included in depicting 
the last -mentioned movement actions. 

[0039] Referring now to Fig. 21 there is illustrated an 
input device 1 including a force/moment sensor in the form 
of an optoelectronic assembly as shown in Fig. 22 
diagrammatically . In Fig. 21 the input device 1 as shown 
therein comprises a circular cylindrical control cap 8 in 
the embodiment as shown, featuring a domed control surface 
9, shown partly cut open in Fig. 21 to expose part of the 
configuration of a force/moment sensor in the form of the 
optoelectronic assembly as shown diagrammatically. 

[0040] Arranged in the optoelectronic assembly as shown 
diagrammatically in Fig. 22 are six light-emitting means 2-1 
to 2-6, preferably in the form of six light-emitting diodes 
in a single plane. Assigned to the light -emitting means 2-1 
to 2-6 fixedly spaced away therefrom in each case is a 
slotted aperture 3-1 to 3-6. In this arrangement, the 
adjacent slotted apertures are configured in a cylindrical 
first ring 3 each offset 90° to the other relative to their 



major direction of extension so that, for example, the 
slotted aperture 3-1 is oriented horizontally and the two 
adjacent slotted apertures 3-2 and 3-6 are oriented 
vertically - 

[0041] In the optoelectronic assembly as shown in Fig, 22 
six position-sensing, photosensitive detectors 4-1 to 4-6 
are arranged in a cylindrical second ring 5 correspondingly 
oriented as regards the assigned slotted apertures 3-1 to 3- 
6. The second ring 5 including the six position-sensing 
detectors 4-1 to 4-6 is furthermore fixedly secured to the 
inner side of the cylindrical control cap 8 of the input 
device 1 . 

[0042] Since the six light -emitting means 2-1 to 2-6 are 
accommodated in a supporting means 6 indicated 
diagrammatically in Fig. 22 as a cylinder which is in turn 
arranged stationary in the input device 1, the second ring 5 
including the attached six position-sensing detectors 4-1 to 
4-6 fixedly secured to the control cap 8 is movable relative 
to the stationary arrangement of the six light-emitting 
means 2-1 to 2-6 and the slotted aperture first ring 3 
fixedly assigned thereto. 

[0043] In one practical embodiment, the stationary 
supporting means 6 indicated diagrammatically as a cylinder 
in Fig. 22 may also be configured as a stationary disk whose 
diameter roughly corresponds to the outer diameter of the 
second ring 5 mounting the position-sensing detectors 4-1 to 
4-6 and which may be arranged above or below the second ring 
5 . 

[0044] Provided between the second ring 5 mounting the 
position-sensing detectors 4-1 to 4-6 and one such disk-type 
supporting means are spring elements 7 (see Fig. 21) in the 
form of coil springs fixedly assigned via thru-bolts (not 
shown) to both the second ring 5 and the supporting means 6. 



I 
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[0045] These coil springs 7 permit movement of the second 
ring 5 mounting the detectors 4-1 to 4-6 via the control cap 
8 relative to the stationary assembly of the light -emitting 
diodes 2-1 to 2-6 and the slotted aperture first ring 3 
fixedly assigned thereto in the direction of the three axes 
X, y, Z of a rectangular three-dimensional system of 
coordinates (see Fig. 1) and about these three axes, and the 
return to the starting position each time after a linear 
displacement or rotational excursion. 

[0046] The six light-emitting diodes 2-1 to 2-6 are 
arranged equally spaced from each other angularly, i.e. at 
an angle of 60° in a single plane and comprise the slotted 
apertures 3-1 to 3-6 oriented alternatingly horizontally and 
vertically to this plane. As evident from the location of 
the individual slotted apertures 3-1 to 3-6 and of the 
planes emanating from the individual diodes 2-1 to 2-6, as 
highlighted shaded, the axes of the individual position- 
sensing detectors 4-1 to 4-6 are always oriented 
perpendicular to the slotted apertures 3-1 to 3-6 assigned 
thereto . 

[0047] The optoelectronic assembly permits total sensing 
of all six possible motional components, namely the three 
translational motions in the direction of the three axes X, 
Y and Z (see Fig. 1) of a rectangular three-dimensional 
system of coordinates and the three rotational motions A, B 
and C about these three axes X, Y and Z. 

[0048] Since the control cap 8 is fixedly secured to the 
second ring 5 mounting the detectors 4-1 to 4-6 and this 
second ring 5 is springingly connected by means of the 
spring elements 7 (Fig. 21) to the stationary supporting 
means 6 mounting the six light -emitting means 2-1 to 2-6 and 
the slotted apertures 3-1 to 3-6 assigned thereto, the 
spring elements 7 maintain the complete measuring system in 



the mechanical zero position when no commands are applied to 
the control cap 8 by the operator. 

[004 9] In this arrangement, by varying the spring 
properties (more particularly their stiffness) the 
operational characteristic of the control cap 8 can be 
influenced within broad limits. When making use of 
relatively soft spring elements 7 the input device 1 has 
more the effect of a path-sensitive sensor, whilst when 
using harder spring elements 7 it is more suitable for 
making commands by exerting forces and moments. 

[0050] Furthermore, each position-sensing detector in the 
optoelectronic assembly is assigned its own light source 
which is signalled by a simple electronic closed loop 
controller, with the aid of which, for example, any 
differences in the sensitivity of the detectors, LED 
efficiency, tolerances in the electronic components, as well 
as temperature drift, are all automatically and speedily 
compensated, thus eliminating the need for any additional 
calibration . 



